1. Field of the Invention
The present invention relates to a correction data generation method and imaging apparatus.
2. Description of the Related Art
Along with the development of LSI techniques, two-dimensional (2D) image processes using digital filters have been performed widely. The 2D image processes include a low-pass filter process, high-pass filter process, convolution filter process, and the like. In this convolution filter process, image data of N×N (N is an integer equal to or larger than 2) pixels having a pixel to be processed as the center are multiplied by predetermined weighting coefficients, and the products are added to form image data of the pixel to be processed.
When a digital filter is designed in correspondence with the characteristics of an optical system, a PSF (Point Spread Function) of the optical system is obtained by experiments or calculations in optical design, and a filter is designed to complement the characteristics. Normally, rotation asymmetric components (coma, astigmatism) of the aberration of the optical system increase as the distance from the optical axis becomes larger. In an ideal inverse filter, a rotation asymmetrical digital filter is formed if rotation asymmetry of the PSF is taken into consideration.
It is impractical for a rotation asymmetrical digital filter to have data as different parameters for respective locations of an image, since each pixel must have data with a matrix size of a filter. For example, when an image consists of M×M pixels, the total number of coefficients in the aforementioned N×N sharpen filter amounts to M2×N2, thus requiring a large memory.
An arrangement that can simultaneously implement geometric conversion and sharpness recovery in a filter based on a pipeline process is disclosed in Japanese Patent No. 3,035,992. This arrangement does not consider rotation symmetry, and must have different coefficient for respective locations.
In consideration of practical processing cost (circuit scale, processing speed), it is preferable to approximate the PSF to a function which has rotation symmetry. As will be described below, a parametric digital filter can be formed by a relatively simple method. To attain a normal use purpose of an imaging device, it is effective to design an inverse filter within a range in which the PSF can be considered to have rotation symmetry, in terms of recovery of frequency components.
In order to parametrically define a digital filter, some methods are available. In an example of a frequency recovery filter having a 3×3 kernel size, a high-frequency emphasis filter H is set by:
                    H        =                  [                                                                      -                  0.5                                                                              -                  1                                                                              -                  0.5                                                                                                      -                  1                                                            7                                                              -                  1                                                                                                      -                  0.5                                                                              -                  1                                                                              -                  0.5                                                              ]                                    (        1        )            
A parametric recovery filter that adjusts the distribution coefficients of this filter H is given by:H0=αI+(1−α)H  (2)
            I:        ⁢                  ⁢    unit    ⁢                  ⁢    matrix    =      [                            0                          0                          0                                      0                          1                          0                                      0                          0                          0                      ]  If α is changed, the characteristics of the recovery filter can be adjusted, and a higher-frequency emphasis filter is obtained with decreasing α.
As another method of designing a parametric filter, a method of approximating the PSF by an exponential function using some criteria of the approximation, and calculating an optimal inverse filter by the method of least squares with respect to that exp function is used. An exponential function is given by:
                                                                        H                ⁡                                  (                                      z                                          i                      ,                      j                                                        )                                            =                                                ∑                                      i                    ,                                          j                      ∈                      E                                                                      ⁢                                                      k                                          i                      ,                      j                                                        ⁢                                      z                                          i                      ,                      j                                                                                                                                                              k                                  i                  ,                  j                                            =                                                1                  I                                ⁢                                  exp                  ⁡                                      (                                                                  -                        d                                            ·                      r                                        )                                                                                                                          r              =                                                                    i                    2                                    +                                      j                    2                                                                                                                          I              =                                                ∑                                      i                    ,                                          j                      ∈                      E                                                                      ⁢                                  exp                  ⁡                                      (                                                                  -                        d                                            ·                      r                                        )                                                                                                          (        3        )            where i and j are indices when the central position of an axis symmetric PSF is expressed by (0, 0). Coefficients of an inverse filter are calculated in correspondence with a change in parameter d. If d is small, the spread of the blur is large, and the high-frequency emphasis effect of the inverse filter is large. In order to suppress errors (aliasing) in a high-frequency range, the high-frequency range may be relaxed without using a δ function as an objective function for the inverse filter. As a design method of a general inverse filter, a calculation example of a least square filter will be described below. In the following example, a target response function is a δ function.
If h(i, j) represents a PSF defined by:region i, j∥i|≦P, |j|≦Q,|a coefficient matrix obtained by expressing this function using a linear filter is given by:
                    h        =                  [                                                                      h                  ⁡                                      (                                                                  -                        P                                            ,                                              -                        Q                                                              )                                                                              ⋯                                                              h                  ⁡                                      (                                                                  -                        P                                            ,                      0                                        )                                                                              ⋯                                                              h                  ⁡                                      (                                                                  -                        P                                            ,                      Q                                        )                                                                                                      ⋮                                            ⋰                                            ⋮                                            ⋰                                            ⋮                                                                                      h                  ⁡                                      (                                          0                      ,                                              -                        Q                                                              )                                                                                                                                                                              h                  ⁡                                      (                                          0                      ,                      0                                        )                                                                                                                                                                              h                  ⁡                                      (                                          0                      ,                      Q                                        )                                                                                                      ⋮                                            ⋰                                            ⋮                                            ⋰                                            ⋮                                                                                      h                  ⁡                                      (                                          P                      ,                                              -                        Q                                                              )                                                                              ⋯                                                              h                  ⁡                                      (                                          P                      ,                      0                                        )                                                                              ⋯                                                              h                  ⁡                                      (                                          P                      ,                      Q                                        )                                                                                ]                                    (        4        )            The Z-transform of the transfer function of the PSF with respect to impulse is given by:
                              Y          ⁡                      (            z            )                          =                              Y            ⁡                          (                                                z                  1                                ,                                  z                  2                                            )                                =                                                    H                ⁡                                  (                                                            z                      1                                        ,                                          z                      2                                                        )                                            ⁢                              δ                ⁡                                  (                                      0                    ,                    0                                    )                                                      =                                          ∑                                  i                  ,                                      j                    =                                          -                      P                                                        ,                                      -                    Q                                                                    P                  ,                  Q                                            ⁢                                                          ⁢                                                                    h                    ⁡                                          (                                              i                        ,                        j                                            )                                                        ·                                      z                    1                                          -                      i                                                                      ⁢                                  z                  2                                      -                    j                                                                                                          (        5        )            Also, an FIR inverse filter to be designed is defined by:
  f  =      [                                        f            ⁡                          (                                                -                  P                                ,                                  -                  Q                                            )                                                ⋯                                      f            ⁡                          (                                                -                  P                                ,                0                            )                                                ⋯                                      f            ⁡                          (                                                -                  P                                ,                Q                            )                                                            ⋮                          ⋰                          ⋮                          ⋰                          ⋮                                                  f            ⁡                          (                              0                ,                                  -                  Q                                            )                                                                                                            f            ⁡                          (                              0                ,                0                            )                                                                                                            f            ⁡                          (                              0                ,                Q                            )                                                            ⋮                          ⋰                          ⋮                          ⋰                          ⋮                                                  f            ⁡                          (                              P                ,                                  -                  Q                                            )                                                ⋯                                      f            ⁡                          (                              P                ,                0                            )                                                ⋯                                      f            ⁡                          (                              P                ,                Q                            )                                            ]  Applying the PSF to an inverse filter yields:
                              r          ⁡                      (                          k              ,              l                        )                          =                              ∑                          i              =                              -                P                                      P                    ⁢                                          ⁢                                    ∑                              j                =                                  -                  Q                                            Q                        ⁢                                                  ⁢                                          h                ⁡                                  (                                                            k                      -                      i                                        ,                                          l                      -                      j                                                        )                                            ⁢                              f                ⁡                                  (                                      k                    ,                    l                                    )                                                                                        (        6        )            Based on approximated least square errors, filter f is given by:
                                          ∑                          i              =                              -                m                                      m                    ⁢                                          ⁢                                    ∑                              j                =                                  -                  m                                            m                        ⁢                                                  ⁢                                          a                ⁡                                  (                                                            p                      -                      i                                        ,                                          q                      -                      j                                                        )                                            ·                              f                ⁡                                  (                                      i                    ,                    j                                    )                                                                    =                                            h              ⁡                              (                                                      -                    p                                    ,                                      -                    q                                                  )                                      ⁢                          a              ⁡                              (                                                      p                    -                    i                                    ,                                      q                    -                    j                                                  )                                              =                                    ∑                              k                =                                  -                  m                                            m                        ⁢                                                  ⁢                                          ∑                                  l                  =                                      -                    m                                                  m                            ⁢                                                          ⁢                                                h                  ⁡                                      (                                          k                      ,                      l                                        )                                                  ·                                  h                  ⁡                                      (                                                                  k                        +                        p                        -                        i                                            ,                                              l                        +                        q                        -                        j                                                              )                                                                                                          (        7        )            Matrix expression of equation (14) for P=Q=m for the sake of simplicity yields:Ax=b  (8)Assume that the contents of x and b have (2m+1)×(2m+1)=4m2+4m+1 elements:
                    b        =                              [                                                                                                                                                                                                                  p                              =                                                              -                                m                                                                                      ,                                                          q                              =                                                              -                                m                                                                                                                                                                                                        h                                ⁡                                                                  (                                                                      m                                    ,                                    m                                                                    )                                                                                            ︸                                                        ,                                                                          ⁢                                                                                                  ⁢                                                                                                            p                              =                                                              -                                m                                                                                      ,                                                          q                              =                                                                                                -                                  m                                                                +                                1                                                                                                                                                                                                                                ⁢                                                                                                                            h                                  ⁡                                                                      (                                                                          m                                      ,                                                                              m                                        -                                        1                                                                                                              )                                                                                                  ︸                                                            ,                                                                                                                          ⁢                              …                              ⁢                                                                                                                          ,                                                                                                      ⁢                                                                                                  ⁢                                                                                                            p                              =                                                              -                                m                                                                                      ,                                                          q                              =                              m                                                                                                                                                                          h                                ⁡                                                                  (                                                                      m                                    ,                                    m                                                                    )                                                                                            ︸                                                        ,                                                                                                                      ︸                                                                              2                            ⁢                            m                                                    +                                                      1                            ⁢                                                                                                                  ⁢                            elements                                                                                                                ⁢                                                                                  ⁢                                                                                                                                                      p                              =                                                                                                -                                  m                                                                +                                1                                                                                      ,                                                          q                              =                                                              -                                m                                                                                                                                                                                                                                ⁢                                                                                                                            h                                  ⁡                                                                      (                                                                                                                  m                                        -                                        1                                                                            ,                                      m                                                                        )                                                                                                  ︸                                                            ,                                                                                                                          ⁢                              …                              ⁢                                                                                                                          ,                                                                                                      ⁢                                                                                                  ⁢                                                                                                            p                              =                                                                                                -                                  m                                                                +                                1                                                                                      ,                                                          q                              =                              m                                                                                                                                                                                                  ⁢                                                                                                                            h                                  ⁡                                                                      (                                                                                                                  m                                        -                                        1                                                                            ,                                      m                                                                        )                                                                                                  ︸                                                            ,                                                                                                                          ⁢                              …                              ⁢                                                                                                                          ,                                                                                                                                                  ︸                                                                              2                            ⁢                            m                                                    +                                                      1                            ⁢                                                                                                                  ⁢                            elements                                                                                                                                                                                                                                                                                                                                                  p                              =                              m                                                        ,                                                          q                              =                                                              -                                m                                                                                                              ⁢                                                                                                                                                                                                                    h                              ⁡                                                              (                                                                                                      -                                    m                                                                    ,                                  m                                                                )                                                                                      ︸                                                    ⁢                                                                                                          ,                                                                                                          ⁢                          …                          ⁢                                                                                                          ,                                                                    ⁢                                                                                                    p                            =                            m                                                    ,                                                      q                            =                            m                                                                                                                                                              h                              ⁡                                                              (                                                                                                      -                                    m                                                                    ,                                                                      -                                    m                                                                                                  )                                                                                      ︸                                                    ,                                                                                                            ︸                                                                        2                          ⁢                          m                                                +                                                  1                          ⁢                                                                                                          ⁢                          elements                                                                                                                                          ]                    T                                    (        9        )                                x        =                              [                                                                                                                                                                                    p                            =                                                          -                              m                                                                                ,                                                      q                            =                                                          -                              m                                                                                                                                                            f                            ⁡                                                          (                                                                                                -                                  m                                                                ,                                                                  -                                  m                                                                                            )                                                                                ︸                                                                    ⁢                                                                                          ⁢                                                                                                    p                            =                                                          -                              m                                                                                ,                                                      q                            =                                                                                          -                                m                                                            +                              1                                                                                                                                                                                          h                              ⁡                                                              (                                                                                                      -                                    m                                                                    ,                                                                                                            -                                      m                                                                        +                                    1                                                                                                  )                                                                                      ︸                                                    ,                                                                                                          ⁢                          …                          ⁢                                                                                                          ,                                                                    ⁢                                                                                          ⁢                                                                                                    p                            =                                                          -                              m                                                                                ,                                                      q                            =                            m                                                                                                                                                              f                              ⁡                                                              (                                                                                                      -                                    m                                                                    ,                                  m                                                                )                                                                                      ︸                                                    ,                                                                                                            ︸                                                                        2                          ⁢                          m                                                +                                                  1                          ⁢                                                                                                          ⁢                          elements                                                                                                                                                                                                                                                                                                                        p                              =                                                                                                -                                  m                                                                +                                1                                                                                      ,                                                          q                              =                                                              -                                m                                                                                                                                                                                                        f                                ⁡                                                                  (                                                                                                                                                    -                                        m                                                                            +                                      1                                                                        ,                                                                          -                                      m                                                                                                        )                                                                                            ︸                                                        ,                                                                                                                  ⁢                            …                            ⁢                                                                                                                  ,                                                                          ⁢                                                                                                  ⁢                                                                                                            p                              =                                                                                                -                                  m                                                                +                                1                                                                                      ,                                                          q                              =                              m                                                                                                                                                                          f                                ⁡                                                                  (                                                                                                                                                    -                                        m                                                                            +                                      1                                                                        ,                                    m                                                                    )                                                                                            ︸                                                        ,                                                                                                                  ⁢                            …                            ⁢                                                                                                                  ,                                                                                                                      ︸                                                                              2                            ⁢                            m                                                    +                                                      1                            ⁢                                                                                                                  ⁢                            elements                                                                                                                ⁢                                                                                  ⁢                                                                                                                                                      p                              =                              m                                                        ,                                                          q                              =                                                              -                                m                                                                                                                                                                                                        f                                ⁡                                                                  (                                                                      m                                    ,                                                                          -                                      m                                                                                                        )                                                                                            ︸                                                        ,                                                                                                                  ⁢                            …                            ⁢                                                                                                                  ,                                                                          ⁢                                                                                                  ⁢                                                                                                            p                              =                              m                                                        ,                                                          q                              =                              m                                                                                                                                          f                              ⁡                                                              (                                                                  m                                  ,                                  m                                                                )                                                                                      ︸                                                                                                                      ︸                                                                              2                            ⁢                            m                                                    +                                                      1                            ⁢                                                                                                                  ⁢                            elements                                                                                                                                                                    ]                    T                                    (        10        )            
Such inverse filter of the exponential function can be used as a parametric sharpen filter, the degree of sharpening of which can be varied by designating the value d in equations (3). Hence, when distances (image heights) from the center of the imager plane are assigned in accordance with pixel positions on the imager plane, and the values d are designated in correspondence with the image heights, shift-variant sharpness recovery can be implemented. Compared to the aforementioned sharpen filter having asymmetric coefficients, the number of coefficients to be stored is overwhelmingly smaller in such parametric sharpen filter.
However, when the inverse filter is limited to the range in which the PSF can be approximated by a rotation symmetric function, as described above, if off-axis astigmatism is large like in, especially, a single-lens, wide-angle imaging system, the image quality can be improved only near the axis.
It is, therefore, an object of the present invention to provide a correction data generation method and imaging apparatus, which can compensate for deterioration of off-axis optical performance without using any rotation asymmetric sharpness recovery filter.